Design and Scaffolded Training of an Efficient DNN Operator for Computer Vision on the Edge

Massively parallel systolic arrays and resource-efficient depthwise separable convolutions are two promising hardware software techniques to accelerate DNN inference on the edge. Interestingly, their combination is inefficient: Computational patterns of do not exhibit a rhythmic flow lack sufficient data reuse saturate arrays. In this article, we formally analyse inefficiency propose an efficient operator, optimal dataflow, superior training methodology towards alleviating this. The called Fully-Separable Convolutions (FuSeConv) , 1 drop-in replacement for depthwise-separable convolutions. FuSeConv generalizes factorization convolution fully along spatial depth dimensions. resultant computation efficiently maps Spatial-Tiled Output Stationary (ST-OS) maximizes efficiency It independent rows array maximise resource-utilization with negligible VLSI overheads. Neural Operator Scaffolding (NOS) scaffolds operators by distilling knowledge from more expensive operation. This bridges accuracy gap between networks Additionally, NOS can be combined Architecture Search (NAS) trade off latency accuracy. hardware-software co-design ST-OS achieves significant speedup 4.1-9.25× state-of-the-art ImageNet dataset. parameter its superiority over illustrates promise as strong solution Training comparable baselines. Further, combining NAS, design that define models improving both computer vision